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ABSTRACT 


Lack of valid means of measuring explicit and implicit knowledge in acquisition of second 
language is a concern issue in investigations of explicit and implicit learning. This paper 
endeavors to validate the use of four tests (i.e.. Untimed Judgment Grammatical Test, 
UJGT; Test of Metalinguistic Knowledge, TMK; Elicited Oral Imitation Test, EOIT; and Time 
pressured Judgment Grammatical Test, TJGT) from a set of instruments introduced by Ellis 
(2005) to measure explicit and implicit knowledge of ESL learners. The result of Principal 
Component Analysis shows the UJGT and TMK loaded on the first factor (i.e., explicit 
knowledge) and the EOIT and TJGT loaded on the second factor (i.e., implicit knowledge) 
when a two factor solution was imposed. The study also shows that second language 
students respond in a different way to ungrammatical and grammatical sentences in the 
UJGT. Hence, Pearson Product Moment Coefficient tests have been conducted amid the 
ungrammatical and grammatical sentences in the UJGT and other instruments. The 
outcome suggests that in the case of UJGT ungrammatical sentences would provide a 
superior measurement of explicit knowledge. 


Keywords: fSL, SL, implicit knowledge, explicit knowledge, test measurement 


INTRODUCTION 

THE terms implicit and explicit knowledge have been applied to language knowledge originally by 
Bialystok (1978). Bialystok suggests that ultimate language fluency and acquisition largely depends on the 
amount of implicit language knowledge or knowledge of a language one has. Explicit language knowledge or 
knowledge about a language represents the conscious facts that can be articulated about the language. Her 
definition indicates similarities between the concepts of Krashen's (1981) "learned system", Anderson's 
(1993) "declarative knowledge" and Langacker's (1991) "external grammar" with explicit language 
knowledge. These are all characterized by awareness of the language knowledge that comes through 
analyzing the language. Similarly, "acquired system", "procedural knowledge", and "internal grammar" are 
comparable to implicit language knowledge, and can be characterized as fluent and accurate language use 
which comes about without thinking or analyzing that knowledge. 

Dispute over the interaction between implicit and explicit knowledge or whether explicit knowledge 
transforms to implicit knowledge in cognitive psychology is recognized as the issue of interface with three 
specific views on instructing grammar: the non-interface view whose advocates believe that explicit 
knowledge cannot transform into implicit knowledge of grammar; the strong interface view whose advocates 
believe that explicit knowledge can transform into implicit knowledge of grammar; and the weak interface 
view whose advocates believe that explicit knowledge can transform into implicit knowledge of grammar in 
certain circumstances and under certain restrictions (Dalili, 2011; Spada, 2015). 
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Some scholars such as Krashen (1982) debated in favor of the non-interface view, while others, such 
as DeKeyser (1998) reinforced the strong interface view. The criticisms directed at both views led to the 
emergence of the integrative view which is recognized as the weak interface view (Ellis, 1994). According to 
this theoretical perspective, it seems that "explicit knowledge by assisting learners to notice linguistic forms 
of input and make a comparison between what they have noticed and their own current interlanguage (i.e., 
by noticing the gap) contributes indirectly to the development of implicit knowledge" (Ellis, 2008). 

Second language acquisition (SLA) researchers' interest in interface studies and debates whether 
explicit knowledge of grammatical forms have a facilitating role or convert to implicit knowledge highlighted 
the necessity of measuring implicit and explicit knowledge separately. 

Yet, pure measurements of either implicit or explicit knowledge currently do not exist (Akakura, 
2009). "Recent experimental developments in measuring language knowledge, however, have enabled closer 
approximations in discriminating between implicit and explicit knowledge" (Ellis, 2005, cited in 
Rohollahzadeh Ebadi, Mohd Saad, & Abedalaziz,, 2014b, p. 26). 

Building on the study of Han and Ellis (1998), Ellis (2005) sought to develop a battery of instruments 
that would make available moderately distinct measurements of explicit and implicit knowledge and 
incorporate a measure of target structures in natural, unplanned language use. Ellis first hypothesized 
behavioral measures differentiating the two knowledge types. Three criteria hypothesized to translate into 
how the tests could be created so as to probabilistically obtain indications of the degree of the two knowledge 
types were: the amount of time available, with time pressure (implicit) vs. no pressure (explicit), the focus of 
attention, with primary focus on meaning (implicit) vs. primary focus on form (explicit) and the utility of 
metalanguage, not required (implicit) vs. encouraged (explicit). Additional conditions were hypothesized to 
provide supporting evidence that the test was in fact measuring what it purported to measure. These were: 
the degree of awareness, responses by feel (implicit) vs. responses by rule (explicit); systematicity, consistent 
responses (implicit) vs. variable responses (explicit); and the degree of certainty in response, high (implicit) 
vs. low (explicit). Learnability, related to the notion of a maturational factor in SLA that is age dependent 
(Long, 2007; Singleton & Ryan, 2004), was also cited as an observed tendency, with early learning favored 
(implicit) vs. later form-focused instruction favored (explicit). These criteria are summarized in Table 1. 


Table 1 Criteria for Measuring Explicit and Implicit Knowledge 


Criterion 

Explicit knowledge 

Implicit 

knowledge 

Current understanding 

Primary focus of 
attention 

Form 

Meaning 

Empirical support (Ellis et al., 2009) 

Time available 

Unrestricted 

Restricted 

Empirical support (Ellis, 2005; Han & 
Ellis, 1998). 

Metalinguistic 

knowledge 

Encouraged 

Not required 

Theoretical support (Elder & 
Manwaring, 2004) 

Degree of awareness 

Response according 

Response 

Unreliable as dependent on self- 

to rule 

according to feel 

report 

Systematicity of 
response 

Variable 

Consistent 

Empirical evidence for variable 
explicit knowledge (Han & Ellis, 

1998). 

Degree of certainty 
in response 

Low 

High 

Empirically unsupported (Ellis, 2005; 
Roehr, 2006) 

Learnability 

Late explicit 

Early learning 

Theoretical support (Long, 2007; 

instruction favored 

favored 

Singleton & Ryan, 2004) 


Note. (Adapted from Ellis, 2005, p. 152) 


Several researchers (Ellis, 2005; Bowles, 2011; Ellis & Loewen, 2007) then explored the extent to 
which it is conceivable to differentiate implicit from explicit knowledge on the basis of behavioral measures 
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hypothesized to distinguish between the two knowledge types. In a study among 91 second language (L2) 
participants and 20 first language participants, knowledge of 17 English constructions deemed difficult by L2 
users was examined using a set of five tests consisting of "an elicited oral imitation test, oral narrative test, 
timed grammaticality judgment test, untimed grammaticality judgment test, and a metalinguistic knowledge 
test" (Ellis, 2005, p. 156). Test scores were analyzed to determine whether there are two underlying 
dimensions (explicit and implicit) in knowledge of second language. A confirmatory factor analysis revealed 
that there were indeed two separate factors these tests loaded onto. The two oral tests (imitation/ narrative) 
and "the timed grammaticality judgment test" which required the unplanned language use under speeded 
conditions loaded on one factor. "The untimed grammaticality judgment test and metalinguistic tests" which 
were expected to be representative of analyzing explicit knowledge loaded on another (Ellis, 2005, p. 161). 

The significance of this result is that it confirmed it is conceivable to measure explicit and implicit 
knowledge separately by manipulating the conditions to elicit one type of language knowledge over the 
other. Ellis and Loewen (2007) and Bowles (2011) in separate studies confirmed and supported Ellis's results. 

Using multiple measures of explicit as well as implicit knowledge was deemed necessary to avoid 
making erroneous inferences (VanPatten & Sanz, 1995), especially since no pure measures of explicit and 
implicit L2 knowledge are possible. 

The goal of this study was to validate the Elicited Oral Imitation Test (EOIT), Time pressured Judgment 
Grammatical Test (TJGT), Untimed Judgment Grammatical Test (UJGT), and Test of Metalinguistic Knowledge 
(TMK) from a set of tests represented in Ellis (2005) to make available relatively separate measures of explicit 
and implicit L2 knowledge following Ellis's guidelines in Malaysia. 

Thus, this research attempts to find out whether the EOIT and TJGT are valid tests for measuring 
implicit knowledge of ESL learners in Malaysia and also whether the UJGT and TMK are valid tests for 
measuring explicit knowledge of ESL learners in Malaysia. Hence the specific question motivating the study 
is: 


• Do scores of the EOIT, TJGT, UJGT and TMK load on two factors, and provide construct validity for 
the tests to measure explicit and implicit L2 knowledge of English learners in the context of Malaysia, 
in the way that they did in R. Ellis (2005)? 


METHOD 
Research context 

This study was done in a selected academic center in Kuala Lumpur in March 2013. While 
there is no consensus among scholars about the appropriate size of a sample in Principal Component Analysis 
(PCA), which is performed in this study, there is a suggestion to have at least 100 +-10 cases for PCA (Hatch 
& Lazaraton, 1991). Thus, we administered the instruments to a group of 94 postgraduate ESL students 
having the same language proficiency level. 

A background questionnaire revealed that most of the L2 users (87%) were from East Asian origin 
with the mean age of 29 years who were staying more than six months in Malaysia. 

Content of the Tests 

The tests in this study were designed and adapted from a set of instruments (Ellis et al., 2009) to 
provide appropriate measurement of learners' L2 knowledge of six grammatical structures. The specific 
structures were selected primarily based on the judgments of ten scholars in the field of Linguistics and SLA 
from "a list of universally problematic structures to learners" (Ellis et al., 2009, pp. 43-44), using a five-point 
Likert Scale. The rating scale of the questionnaire determined the number of problematic structures to be 
used as specific structures of the study. Finally, Regular past tense, Modal verbs (can, have to). Unreal 
conditionals. Present perfect tense and Comparative adjectives were selected (Rohollahzadeh Ebadi, 
Abedalaziz, Mohd Saad, & Chin, 2014). 

Description of the Tests 

(a) Elicited Oral Imitation Test (EOIT). The EOIT consists of 24 belief sentences. Half of the sentences 
(i.e., 12 statements) are grammatically correct and the other statements are grammatically incorrect. In this 
test four statements, two grammatical and two ungrammatical, are allocated to each specific structure. Using 
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audio tape, each sentence one by one was presented orally: To focus students' attention on the meaning, 
they were required first to indicate whether the sentence is True or False, or whether they are not sure about 
it. Then, students were required to restate the correct form of the sentence verbally. Learners' answers to 
the statements were recorded by audiotape to analyze and find out whether a specific structure had been 
applied. Each reproduced sentence was assigned a mark either one (if the target structure was correctly 
reproduced) or zero (if it was avoided or incorrectly imitated). 

(b) Time pressured Judgment Grammatical Test (TJGT). The TJGT consists of 36 sentences, 18 
grammatical and 18 ungrammatical sentences. Six sentences (i.e., three grammatical and three 
ungrammatical sentences) have been allocated for each target structure of the study. The test was delivered 
on the computer screen within a specific time limit, using PowerPoint slides for each student in computer 
labs. Students were required to judge whether each sentence is ungrammatical or grammatical by marking 
the correct response in their answer sheets. "The time limit for each sentence was specified on the basis of 
average response time by native speakers. Considering slower processing speed of ESL learners, 30% of [the] 
time taken for each sentence was added" (Rohollahzadeh Ebadi, Mohd Saad, & Abedalaziz, 2014a, p. 14). 
Each item of the test was scored either correct or incorrect, with unanswered items marked as incorrect. 

(c) Untimed Judgment Grammatical Test (UJGT). The UJGT included the same types of sentences as 
the TJGT but in different order. It was also delivered in written form but on paper. Students were asked to 
point out whether each statement is True or False, just as they had done in the TJGT, except that they were 
instructed to answer at their own pace because the test had no set time limit. 

(d) Test of Metalinguistic Knowledge (TMK). This test is based on the test in Alderson, Clapham, and 
Steel (1997) as reported in Ellis et al. (2009). This is an untimed test in two sections. The first section included 
12 ungrammatical sentences based on the target structures of the study. Each sentence contains a 
grammatical error which is underlined. Students are asked to select the rule, out of 4 provided choices, that 
best explains each error. In the second section students are presented with another 12 sentences. In front of 
each sentence is a bracket within which a grammatical feature is mentioned. Students are asked to find the 
item requested and underline it in the presented sentence. And finally a total percentage accuracy score is 
calculated. 

The EOIT and the TJGT are designed to measure implicit L2 knowledge of our target structures. "These 
tests are designed to elicit the learners 'feel' for what was grammatical, they will require learners to process 
language without encouraging the use of metalinguistic knowledge" (Ellis et al., 2009, p. 46). UJGT and TMK 
are designed to measure explicit L2 knowledge, "favor the use of 'rule' and are unpressured, it will focus 
attention on form and encourage the application of metalinguistic knowledge" (See Table 3) (Ellis et al., 2009, 
P-46). 

Table 3 Features of the Instruments of the Study 


Measure 

EOIT 

TJGT 

UJGT 

TMK 

Awareness degree 

by feel 

by feel 

by rule 

by rule 

Availability of time 

pressured 

pressured 

not pressured not pressured 

Concentration 

on meaning 

on form 

on form 

on form 

Utility of Knowledge of 

no 

no 

yes 

yes 

Metalanguage 






Source. Adapted from Ellis et al., 2009, p. 47 


Data Analysis and Results 

Establishing reliability. Instrument reliability was calculated using Cronbach's alpha coefficient of internal 
consistency (SPSS Version 21). The instruments for measuring implicit L2 knowledge consisted of EOIT and 
TJGT. These tests were conducted under time pressure and their focus of attention was on meaning. The 
instruments for measuring explicit L2 knowledge consisted of UJGT and TMK. In these tests the focus of 
attention was on form and the measures of explicit L2 knowledge were self-paced; participants were given 
no restrictions on the amount of time to reflect on their knowledge. The tests were conducted in a language 
laboratory at the education center for all participants in this order: EOIT, TJGT, UJGT and TMK. This order of 
presentation ensured that the explicit knowledge tests would not prime learners. The four tests took around 
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two hours to complete by participants. The number of target items tested in each test is described in Table 
4. 


Table 4 Number of Target Items Tested in the Study 


Test 

Grammatical 

Items 

Ungrammatical 

Items 

Total 

EOIT 

12 

12 

24 

TJGT 

18 

18 

36 

UJGT 

18 

18 

36 

TMK 

12 

12 

24 


As summarized in Table 5, the reliability values of the four instruments are above .80; therefore 
suggesting very good internal consistency (Pallant, 2010). 


Table 5 Reliability (Cronbach's alpha) Value of Instruments 


Test 

N of 

participants 

N of items 

Cronbach's 

alpha 

EOIT 

94 

24 

.87 

TGJT 

94 

36 

.91 

UGJT 

94 

36 

.82 

MKT 

94 

24 

.86 


Moreover, Item-Total Statistics table of analysis shows correlation value of more than 0.3, indicating 
appropriate correlation of each item with the total score (Pallant, 2010). 

Establishing validity. Evidence for test validity was first investigated by judgment of ten scholars in 
the field of linguistics. 

Second, in order to check for evidence that the tests may tap the types of knowledge as hypothesized, 
construct validity of the instruments was assessed. 

In order to investigate whether the EOIT, TJGT, UJGT and TMK predominantly assessed the types of 
knowledge as hypothesized, a PCA was conducted. "Principal Component Analysis is directed toward 
enabling one to use fewer variables to provide the same information that one would obtain from a larger set 
of variables" (Leech, Barrett, & Morgan, 2005, p. 76). 

Prior to PCA, the researcher visually inspected the bivariate correlation matrix as a preliminary step 
to assess inter-item correlation. The investigation indicates moderate range (.3) and above for most of the 
values. Then, the researcher calculated the Kaiser-Meyer-Olkin (KMO) Measure of Sampling Adequacy. This 
was attained by "a ratio of the sum of the squared correlations to the sum of the squared correlations plus 
squared partial correlations" (Pallant, 2010, p. 181). The result shows The KMO value was acceptable at .794 
indicating factor analysis was appropriate for the scale. Moreover, as presented in Table 6 Bartlett's Test of 
Sphericity was significant [x 2 (11.28) = 2884.824]; hence the null hypothesis that the correlation matrix was 
an identity matrix was not met. 

Table 6 KMO and Bartlett's Test 


Kaiser-Meyer-Olkin Measure of Sampling 
. , .794 

Adequacy._ 

Bartlett's Test of Approx. Chi-Square 2884.824 

Sphericity Df 1128 

Sig.,000 
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"By rejecting the null hypothesis the correlation matrix was deemed acceptable for factor analytic 
techniques" (Pallant, 2010, p. 181). The results shows high communalities extending from .488 to .831, and 
factors with eigenvalues of greater than 1.0, accounting for 69.25% of variance. Inspecting the scree plot (see 
Figure 1), and judging from previous theory, two factors which capture much more of the variance (44.74%) 
were retained. 

Figure 1 shows the Scree plot of the sores in PCA. 


Scree Plot 



Figure 1. The Scree plot in PCA. 

Additionally, the Component Matrix table indicates that most of the items load quite strongly (above 
.4) on the first two components. 

Ideally, an additional Confirmatory Factor Analysis which assumes all associations between factors 
as unanalyzed may also be conducted, as has been pointed out by Isemonger (2007). Flowever, as the 
Principal Components Factor Analysis yielded two factors, a Confirmatory Factor Analysis was not conducted. 

Third, in order to examine the psychometric properties of the ungrammatical and grammatical 
statements in UJGT, as proposed by Ellis and his colleagues (2009), Pearson Product Moment Coefficient 
analysis was conducted. The result is shown in Table 3.7. 

Table 3.7 Correlation Matrix of Grammatical and Ungrammatical Structures in UJGT with Other Three Tests 



TJGT 

UJGT grammatical 

UJGT ungrammatical 

TMK 


structures 

structures 

EOIT 




-.144 

TJGT 



-.029 

-.084 

UJGT 

grammatical structures 

.894* 

.859* 

.008 

.005 

UJGT 

ungrammatical 

structures 


.876* 

.067 

.791* 


* Correlation is significant at the .05 level. 

As shown by the results in Table 3.7 the scores of the grammatical sentences of UJGT correlate 
significantly (at .05 level) with scores of the other tests, but more strongly with scores of the EOIT (r = .859) 
and TJGT (r = .876) than with scores of the TMK (r = .005). In contrast, the ungrammatical sentences' scores 
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correlate very strongly with scores of the TMK (r = .79) and less strongly with scores of the EOIT (r = -.029) 
and TJGT (r = .008). 

CONCLUSION 


This study was primarily concerned with providing evidence for the construct validity of four 
tests from a set of tests designed in Ellis (2005) to make available moderately separate measures of explicit 
and implicit knowledge. The research question addressed the issue of validity generally and asked whether 
scores on the four tests would load on two separate factors, one representing implicit knowledge and the 
other representing explicit knowledge. 

The result of PCA shows that the EOIT and TJGT loaded on one factor (implicit knowledge) and the 
UJGT and TMK loaded on the second factor (explicit knowledge) when a two factor solution was imposed. 
Hence the study supports the theory (Ellis, 2005) that implicit and explicit knowledge each represent separate 
constructs. 

Moreover, the result of Pearson Product Moment Coefficients between the scores of the 
ungrammatical and grammatical statements in UJGT and scores of the other tests shows that the 
grammatical sentences' scores correlate more strongly with the EOIT and TJGT than with TMK. In contrast, 
the ungrammatical sentences' scores correlate very strongly with scores of the TMK and less strongly with 
scores of the EOIT and TJGT. This suggests that in the case of UJGT the scores of the ungrammatical sentences 
would provide a better measure of explicit L2 knowledge than the scores of the grammatical sentences or 
total scores. 

Therefore, this study provides support for the construct validity of the EOIT, TJGT, UJGT and TMK 
from the battery of tests in Ellis (2005), with another population of learners. It gets the field of ESL study one 
step closer to having reliable and valid instruments for measuring implicit and explicit linguistic knowledge 
of learners. Such instruments allow researchers to "investigate issues of central theoretical importance in 
the study of L2 acquisition" (Ellis, 2005, as cited in Bowles, 2011, p. 262). 

Some researchers (e.g., Bowles, 2011; Ellis & Loewen, 2007) suggested that separate measurement 
of explicit and implicit L2 knowledge could provide better understanding of SLA studies. Ellis (2009) said "the 
main limitation of the research to date lies in the method of testing" (p. 315). These could provide 
appropriate measurements to come across the controversial issue of interface and noninterface positions of 
L2 researchers. The study could help to clarify some of the ambiguities and substantiate some of the findings 
which exist in the SLA researches. 

All the same, armed with that knowledge, researchers and teachers will be better equipped to design 
instruction in English classes. They also could determine in what ways various components, such as explicit 
information, negative evidence, and structured input affect the interlanguage system. 

This study was limited in terms of its target structures, participants and so on. Thus, further 
researches would shed more light in this field of study. 
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